fix: restore post-connect fallback for multi-network stacks on API < 1.44 by jotka · Pull Request #13629 · docker/compose

jotka · 2026-03-09T15:33:59Z

Problem

Docker Compose 5.1.0 broke multi-network stacks on Docker daemons older than API 1.44 (Docker Engine < 23.0), including Synology DSM 7.1 (API 1.41) and DSM 7.2 (API 1.43). The error is:

Container cannot be connected to network endpoints

This is a regression from 5.0.2 which worked correctly on these daemons.

Closes #13628

Context

I maintain Dockhand, a Docker management application. Many of our users run Dockhand on Synology NAS devices (DSM 7.1/7.2), which ship with Docker 20.10 (API 1.41). This regression broke multi-network stack deployments for all of them when we updated docker-compose to 5.1.0. We had to cap docker-compose at 5.0.2 as a workaround while investigating the root cause.

Root Cause

Both regressions were introduced in commit bfb5511 (go.mod: bump github.com/moby/moby/api v1.53.0, moby/client v0.2.2):

1. defaultNetworkSettings() in create.go

The versions.GreaterThanOrEqualTo(version, APIVersion144) guard was removed during the moby API bump, making defaultNetworkSettings unconditionally include ALL networks in EndpointsConfig for ContainerCreate. Old daemons (< API 1.44) only accept a single entry and reject the request with the error above.

2. The post-connect fallback was removed from convergence.go

The same commit removed the NetworkConnect fallback loop from createMobyContainer(). Previously (since aaa7ef6), after ContainerCreate, the code checked versions.LessThan(apiVersion, "1.44") and connected extra networks one-by-one via NetworkConnect(). This entire block was dropped during the API migration.

Fix

First commit (`4f14ab8`): Restore backward compat

Restores the two behaviors from 5.0.2:

create.go — defaultNetworkSettings(): Only include extra networks in EndpointsConfig when apiVersion >= 1.44 (guarded by !versions.LessThan(version, apiVersion144)).
convergence.go — createMobyContainer(): Restore the post-connect fallback. After container creation, when apiVersion < 1.44, iterate the service's extra networks and call NetworkConnect() for each, skipping the primary network (already configured via NetworkMode in ContainerCreate).

Second commit (`7895a5a`)

Pass APIVersion through createConfigs: Added APIVersion field to the createConfigs struct, populated from the existing apiVersion local in getCreateConfigs(). This eliminates the redundant s.RuntimeVersion(ctx) call in createMobyContainer.
Reorder ContainerInspect after NetworkConnect: Moved the inspect call after the NetworkConnect loop so the returned container.Summary includes all attached networks.
Unit tests for the < 1.44 code path:
- TestDefaultNetworkSettings sub-test with version "1.43" — asserts EndpointsConfig contains only the primary network.
- TestCreateMobyContainerLegacyAPI — asserts ContainerCreate gets only the primary network, NetworkConnect is called for the secondary, and ContainerInspect runs after NetworkConnect.

Testing

Unit tests pass (the TestRunHook_ConsoleSize failure is a pre-existing PTY crash on macOS/arm64, unrelated to this change and present on main before this fix)
Tested manually against Docker 20.10 (API 1.41) — multi-network stacks now deploy correctly
No regression expected on modern daemons (API >= 1.44): they continue to use the multi-endpoint ContainerCreate path

jotka · 2026-03-23T07:30:11Z

Good catch to flag — I had the same instinct initially. But after scanning the codebase, this comparison is safe due to a structural guarantee in compose-go:

network_mode and networks are mutually exclusive (loader/validate.go:67-68). When network_mode is explicitly set (host, none, container:xxx, service:xxx → resolved to container:<id>), defaultNetworkSettings() returns early with nil NetworkingConfig, so the loop body never executes — there are zero networks to iterate.

The only case where the loop runs is the standard networks: path, where defaultNetworkSettings() sets NetworkMode to exactly container.NetworkMode(project.Networks[primaryKey].Name) — the same moby name string we compare against.

That said, the codebase itself identifies primary by position (serviceNetworks[0] from NetworksByPriority()), not by string match. I can switch to a positional skip if you'd prefer:

for i, networkKey := range serviceNetworks {
    if i == 0 {
        continue // primary, already in ContainerCreate
    }

Functionally equivalent here, but arguably more explicit about intent. Let me know.

Copilot

Pull request overview

Restores Docker Compose’s backward-compatible multi-network container creation behavior for Docker Engine API versions older than 1.44, preventing ContainerCreate failures on legacy daemons (e.g., Docker 20.10 / Synology DSM).

Changes:

Reintroduced API-version gating so ContainerCreate only includes multiple EndpointsConfig entries when apiVersion >= 1.44.
Restored the legacy post-create network attachment path (NetworkConnect) for apiVersion < 1.44, and ensured ContainerInspect happens after legacy network connects.
Added/updated unit tests covering the < 1.44 behavior in both network config generation and container creation.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
pkg/compose/create.go	Adds `APIVersion` to create configs and gates extra `EndpointsConfig` entries behind `apiVersion >= 1.44`.
pkg/compose/convergence.go	Restores legacy `NetworkConnect` loop for `< 1.44` before inspecting the container.
pkg/compose/api_versions.go	Documents/defines the `apiVersion144` threshold and its behavioral implications.
pkg/compose/create_test.go	Adds a subtest asserting only the primary network is included for API `1.43`.
pkg/compose/convergence_test.go	Adds a legacy API test asserting `NetworkConnect` is used and `ContainerInspect` occurs after connects.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

glours

Hey @jotka
Thanks for this fix, unfortunately it could introduced creation of unwanted orphan containers when the process can't create endpoint or attach to networks.

There is also a separate issue where the fallback decision is based on the daemon’s advertised max API version instead of the negotiated client API version; that part has been handled separately in PR #13690.

glours · 2026-03-30T13:41:09Z

pkg/compose/convergence.go

+			epSettings, err := createEndpointSettings(project, service, number, networkKey, cfgs.Links, opts.UseNetworkAliases)
+			if err != nil {
+				return created, err
+			}
+			if _, err := s.apiClient().NetworkConnect(ctx, mobyNetworkName, client.NetworkConnectOptions{
+				Container:      response.ID,
+				EndpointConfig: epSettings,
+			}); err != nil {
+				return created, err


For API < 1.44, this PR now creates the container first and then attaches secondary networks one by one here. If one of these NetworkConnect() calls fails, the function returns immediately and leaves the just-created container behind on the primary network. That means the legacy fallback is no longer atomic on failure, and a later up / recreate can hit a name conflict against the orphaned container.

I think we should clean up the created container on NetworkConnect failure before returning.

Good catch, thanks! Added ContainerRemove with Force:true on both error paths (createEndpointSettings and NetworkConnect). Also added a test for the failure case. Re the API version negotiation — noted, will rebase on top of #13690 once it lands.

FYI PR #13690 has been merged
ps: don't forgot to sign-off your commits 😉

jotka · 2026-03-31T12:56:28Z

Rebased on main (post go1.25.8, moby/client v1.54.0, etc.) and fixed lint issues (line length in test closures). All tests pass locally.

Before API 1.44 (Docker Engine 25.0), ContainerCreate only accepted a single EndpointsConfig entry. Passing multiple entries silently drops all but one network, leaving containers under-connected on older daemons such as Docker 20.10 or Synology DSM 7.1/7.2. Add apiVersion144 constant and wrap the secondary-networks loop in defaultNetworkSettings so that only the primary network is included in the ContainerCreate call when the negotiated API version is below 1.44. Signed-off-by: jarek <jkrochmalski@gmail.com>

For Docker daemons older than API 1.44, the extra networks omitted from ContainerCreate must be connected individually after creation via NetworkConnect. If any NetworkConnect call fails, remove the freshly created container to prevent orphans. Add two tests: - TestCreateMobyContainerLegacyAPI: happy path verifying NetworkConnect is called for the secondary network on API 1.43 - TestCreateMobyContainerLegacyAPI_NetworkConnectFailure: verifies ContainerRemove is called with Force when NetworkConnect fails Signed-off-by: jarek <jkrochmalski@gmail.com>

jotka · 2026-03-31T13:05:11Z

Rebased on main (post #13690 merge). Changes:

Dropped APIVersion from createConfigs struct — no longer needed since createMobyContainer now calls s.RuntimeAPIVersion(ctx) directly
Updated all test mocking to use the new Ping + ClientVersion pattern (no more ServerVersion or package-level runtimeVersion reset)
Fixed lint issues (line length in test closures)

All tests pass, linter clean.

glours

Sounds good to me 👍
Thanks for the contribution
I'll do a release later this week so you can update Dockhand on your side

codecov · 2026-03-31T13:32:56Z

Codecov Report

❌ Patch coverage is 73.07692% with 7 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
pkg/compose/convergence.go	73.68%	3 Missing and 2 partials ⚠️
pkg/compose/create.go	71.42%	1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

jotka · 2026-03-31T13:33:49Z

awesome, thanks!

jotka requested a review from a team as a code owner March 9, 2026 15:34

jotka requested review from glours and ndeloof March 9, 2026 15:34

jotka force-pushed the fix/multi-network-old-daemon-compat branch 3 times, most recently from af22a2b to 7895a5a Compare March 9, 2026 16:47

Copilot AI review requested due to automatic review settings March 23, 2026 17:12

Copilot started reviewing on behalf of jotka March 23, 2026 17:13 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

jotka mentioned this pull request Mar 28, 2026

[bug] regression from 5.02 - multi-network stacks broken on Docker API < 1.44 (regression in v5.1.0) #13628

Closed

glours reviewed Mar 30, 2026

View reviewed changes

jotka force-pushed the fix/multi-network-old-daemon-compat branch from 8e26cb7 to 1e13ed6 Compare March 31, 2026 12:56

jotka added 2 commits March 31, 2026 15:04

jotka force-pushed the fix/multi-network-old-daemon-compat branch from 1e13ed6 to 4e64b82 Compare March 31, 2026 13:05

glours approved these changes Mar 31, 2026

View reviewed changes

glours merged commit a97738d into docker:main Mar 31, 2026
40 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: restore post-connect fallback for multi-network stacks on API < 1.44#13629

fix: restore post-connect fallback for multi-network stacks on API < 1.44#13629
glours merged 2 commits intodocker:mainfrom
jotka:fix/multi-network-old-daemon-compat

jotka commented Mar 9, 2026 •

edited

Loading

Uh oh!

jotka commented Mar 23, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

glours left a comment

Uh oh!

glours Mar 30, 2026

Uh oh!

jotka Mar 30, 2026

Uh oh!

glours Mar 31, 2026

Uh oh!

jotka commented Mar 31, 2026

Uh oh!

jotka commented Mar 31, 2026

Uh oh!

glours left a comment

Uh oh!

codecov bot commented Mar 31, 2026

Uh oh!

jotka commented Mar 31, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jotka commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Context

Root Cause

Fix

First commit (4f14ab8): Restore backward compat

Second commit (7895a5a)

Testing

Uh oh!

jotka commented Mar 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

glours left a comment

Choose a reason for hiding this comment

Uh oh!

glours Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

jotka Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

glours Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

jotka commented Mar 31, 2026

Uh oh!

jotka commented Mar 31, 2026

Uh oh!

glours left a comment

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Mar 31, 2026

Codecov Report

Uh oh!

jotka commented Mar 31, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jotka commented Mar 9, 2026 •

edited

Loading

First commit (`4f14ab8`): Restore backward compat

Second commit (`7895a5a`)